Ill-posed Linear Inverse Problems Research Articles

A new second-order dynamical method for solving linear inverse problems in Hilbert spaces

A new second-order dynamic method (SODM) is proposed for solving ill-posed linear inverse problems in Hilbert spaces. The SODM can be viewed as a combination of Tikhonov regularization and second-order asymptotical regularization methods. As a result, a double-regularization-parameter strategy is adopted. The regularization properties of SODM are demonstrated under both a priori and a posteriori stopping rules. In the context of time discretization, we propose several iterative schemes with different choices of damping parameters. A truncated discrepancy principle is employed as the stop criterion. Finally, numerical experiments are performed to show the efficiency of the SODM: on the whole, compared with the classical Tikhonov method and the first-order dynamical-system method, the SODM leads to more-accurate approximate solutions while requiring fewer-iterative numbers.

Convergent Data-Driven Regularizations for CT Reconstruction

AbstractThe reconstruction of images from their corresponding noisy Radon transform is a typical example of an ill-posed linear inverse problem as arising in the application of computerized tomography (CT). As the (naïve) solution does not depend on the measured data continuously, regularization is needed to reestablish a continuous dependence. In this work, we investigate simple, but yet still provably convergent approaches to learning linear regularization methods from data. More specifically, we analyze two approaches: one generic linear regularization that learns how to manipulate the singular values of the linear operator in an extension of our previous work, and one tailored approach in the Fourier domain that is specific to CT-reconstruction. We prove that such approaches become convergent regularization methods as well as the fact that the reconstructions they provide are typically much smoother than the training data they were trained on. Finally, we compare the spectral as well as the Fourier-based approaches for CT-reconstruction numerically, discuss their advantages and disadvantages and investigate the effect of discretization errors at different resolutions.

Learning a microlocal prior for limited-angle tomography

Abstract Limited-angle tomography is a highly ill-posed linear inverse problem. It arises in many applications, such as digital breast tomosynthesis. Reconstructions from limited-angle data typically suffer from severe stretching of features along the central direction of projections, leading to poor separation between slices perpendicular to the central direction. In this paper, a new method is introduced, based on machine learning and geometry, producing an estimate for interfaces between regions of different X-ray attenuation. The estimate can be presented on top of the reconstruction, indicating more reliably the separation between features. The method uses directional edge detection, implemented using complex wavelets and enhanced with morphological operations. By using convolutional neural networks, the visible part of the singular support is first extracted and then extended to the full domain, filling in the parts of the singular support that would otherwise be hidden due to the lack of measurement directions.

On Convergence Rates of Proximal Alternating Direction Method of Multipliers

In this paper we consider from two different aspects the proximal alternating direction method of multipliers (ADMM) in Hilbert spaces. We first consider the application of the proximal ADMM to solve well-posed linearly constrained two-block separable convex minimization problems in Hilbert spaces and obtain new and improved non-ergodic convergence rate results, including linear and sublinear rates under certain regularity conditions. We next consider the proximal ADMM as a regularization method for solving linear ill-posed inverse problems in Hilbert spaces. When the data is corrupted by additive noise, we establish, under a benchmark source condition, a convergence rate result in terms of the noise level when the number of iterations is properly chosen.

A relaxed iterated Tikhonov regularization for linear ill-posed inverse problems

Iterated Tikhonov regularization methods are prevalent regularization methods that can make the solution of ill-posed problems less sensitive to noise. As the iteration number grows, their solution gradually converges to the optimum approximation of the desired solution but then changes to the worse, causing a semiconvergence. This difficulty can be reduced by a reliable early termination rule or a strategy for choosing the appropriate regularization parameter at each iteration. Different from the existing methods, this paper describes a novel iterative Tikhonov regularization based on introducing a non-decreasing sequence of relaxation parameters into the stationary iterated Tikhonov regularization, providing a fast and roughly stable convergence. After investigating theoretically the convergence rate of the novel iterative Tikhonov regularization, we propose a series of numerical experiments for the evaluation of its accuracy and finally describe a real-world application in the field of 3-dimensional image restoration problems to verify its efficiency.

Evaluation of the Number of Degrees of Freedom of the Field Scattered by a 3D Geometry.

The solution to an ill-posed linear inverse problem requires the use of regularization methods to achieve a stable approximation solution. One powerful approach is the truncated singular value decomposition (TSVD), but it requires an appropriate choice of the truncation level. One suitable option is to take into account the number of degrees of freedom (NDF) of the scattered field, which is defined by the step-like behavior of the singular values of the relevant operator. Then, the NDF can be estimated as the number of singular values preceding the knee or the exponential decay. Therefore, an analytical estimation of the NDF is significant for obtaining a stable, regularized solution. This paper addresses the analytical estimation of the NDF of the field scattered by the surface of a cube geometry for a single frequency and the multi-view case in the far-zone. In addition, a method is proposed to find the minimum numbers of plane waves and their directions to achieve the total estimated NDF. The main results are that the NDF is related to the measure of the surface of the cube and can be achieved by only considering a limited number of impinging plane waves. The efficiency of the theoretical discussion is demonstrated through a reconstruction application for microwave tomography of a dielectric object. Numerical examples are provided to confirm the theoretical results.

Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: application to surgical imaging

PurposeHyperspectral imaging has the potential to improve intraoperative decision making if tissue characterisation is performed in real-time and with high-resolution. Hyperspectral snapshot mosaic sensors offer a promising approach due to their fast acquisition speed and compact size. However, a demosaicking algorithm is required to fully recover the spatial and spectral information of the snapshot images. Most state-of-the-art demosaicking algorithms require ground-truth training data with paired snapshot and high-resolution hyperspectral images, but such imagery pairs with the exact same scene are physically impossible to acquire in intraoperative settings. In this work, we present a fully unsupervised hyperspectral image demosaicking algorithm which only requires exemplar snapshot images for training purposes.MethodsWe regard hyperspectral demosaicking as an ill-posed linear inverse problem which we solve using a deep neural network. We take advantage of the spectral correlation occurring in natural scenes to design a novel inter spectral band regularisation term based on spatial gradient consistency. By combining our proposed term with standard regularisation techniques and exploiting a standard data fidelity term, we obtain an unsupervised loss function for training deep neural networks, which allows us to achieve real-time hyperspectral image demosaicking.ResultsQuantitative results on hyperspetral image datasets show that our unsupervised demosaicking approach can achieve similar performance to its supervised counter-part, and significantly outperform linear demosaicking. A qualitative user study on real snapshot hyperspectral surgical images confirms the results from the quantitative analysis.ConclusionOur results suggest that the proposed unsupervised algorithm can achieve promising hyperspectral demosaicking in real-time thus advancing the suitability of the modality for intraoperative use.

Galaxy image deconvolution for weak gravitational lensing with unrolled plug-and-play ADMM

ABSTRACT Removing optical and atmospheric blur from galaxy images significantly improves galaxy shape measurements for weak gravitational lensing and galaxy evolution studies. This ill-posed linear inverse problem is usually solved with deconvolution algorithms enhanced by regularisation priors or deep learning. We introduce a so-called ’physics-informed deep learning’ approach to the Point Spread Function (PSF) deconvolution problem in galaxy surveys. We apply algorithm unrolling and the Plug-and-Play technique to the Alternating Direction Method of Multipliers (ADMM), in which a neural network learns appropriate hyperparameters and denoising priors from simulated galaxy images. We characterize the time-performance trade-off of several methods for galaxies of differing brightness levels, as well as our method’s robustness to systematic PSF errors and network ablations. We show an improvement in reduced shear ellipticity error of 38.6 per cent (SNR=20)/45.0 per cent (SNR=200) compared to classic methods and 7.4 per cent (SNR=20)/33.2 per cent (SNR = 200) compared to modern methods (https://github.com/Lukeli0425/Galaxy-Deconv).

Inverse learning in Hilbert scales

We study linear ill-posed inverse problems with noisy data in the framework of statistical learning. The corresponding linear operator equation is assumed to fit a given Hilbert scale, generated by some unbounded self-adjoint operator. Approximate reconstructions from random noisy data are obtained with general regularization schemes in such a way that these belong to the domain of the generator. The analysis has thus to distinguish two cases, the regular one, when the true solution also belongs to the domain of the generator, and the ‘oversmoothing’ one, when this is not the case. Rates of convergence for the regularized solutions will be expressed in terms of certain distance functions. For solutions with smoothness given in terms of source conditions with respect to the scale generating operator, then the error bounds can then be made explicit in terms of the sample size.

On Ambiguity in Linear Inverse Problems: Entrywise Bounds on Nearly Data-Consistent Solutions and Entrywise Condition Numbers.

Ill-posed linear inverse problems appear frequently in various signal processing applications. It can be very useful to have theoretical characterizations that quantify the level of ill-posedness for a given inverse problem and the degree of ambiguity that may exist about its solution. Traditional measures of ill-posedness, such as the condition number of a matrix, provide characterizations that are global in nature. While such characterizations can be powerful, they can also fail to provide full insight into situations where certain entries of the solution vector are more or less ambiguous than others. In this work, we derive novel theoretical lower- and upper-bounds that apply to individual entries of the solution vector, and are valid for all potential solution vectors that are nearly data-consistent. These bounds are agnostic to the noise statistics and the specific method used to solve the inverse problem, and are also shown to be tight. In addition, our results also lead us to introduce an entrywise version of the traditional condition number, which provides a substantially more nuanced characterization of scenarios where certain elements of the solution vector are less sensitive to perturbations than others. Our results are illustrated in an application to magnetic resonance imaging reconstruction, and we include discussions of practical computation methods for large-scale inverse problems, connections between our new theory and the traditional Cramér-Rao bound under statistical modeling assumptions, and potential extensions to cases involving constraints beyond just data-consistency.

Distributed spectral pairwise ranking algorithms

This paper considers spectral pairwise ranking algorithms in a reproducing kernel Hilbert space. The concerned algorithms include a large family of regularized pairwise learning algorithms. Motivated by regularization methods, spectral algorithms are proposed to solve ill-posed linear inverse problems, then developed in learning theory and inverse problems. Recently, pairwise learning tasks such as bipartite ranking, similarity metric learning, Minimum Error Entropy principle, and AUC maximization have received increasing attention due to their wide applications. However, the spectral algorithm acts on the spectrum of the empirical integral operator or kernel matrix, involving the singular value decomposition or the inverse of the matrix, which is time-consuming when the sample size is immense. Our contribution is twofold. First, under some general source conditions and capacity assumptions, we establish the first-ever mini-max optimal convergence rates for spectral pairwise ranking algorithms. Second, we consider the distributed version of the algorithms based on a divide-and-conquer approach and show that, as long as the partition of the data set is not too large, the distributed learning algorithm enjoys both computational efficiency and statistical optimality.

On the acceleration of optimal regularization algorithms for linear ill-posed inverse problems

On the acceleration of optimal regularization algorithms for linear ill-posed inverse problems

Sparse Synthetic Aperture Radar Imaging From Compressed Sensing and Machine Learning: Theories, applications, and trends

Synthetic aperture radar (SAR) image formation can be treated as a class of ill-posed linear inverse problems, and the resolution is limited by the data bandwidth for traditional imaging techniques via matched filter (MF). The sparse SAR imaging technology using compressed sensing (CS) has been developed for enhanced performance, such as superresolution, feature enhancement, etc. More recently, sparse SAR imaging from machine learning (ML), including deep learning (DL), has been further studied, showing great potential in the imaging area. However, there are still gaps between the two groups of methods for sparse SAR imaging, and their connections have not been established.

Automatic balancing parameter selection for Tikhonov-TV regularization

This paper considers large-scale linear ill-posed inverse problems whose solutions can be represented as sums of smooth and piecewise constant components. To solve such problems we consider regularizers consisting of two terms that must be balanced. Namely, a Tikhonov term guarantees the smoothness of the smooth solution component, while a total-variation (TV) regularizer promotes blockiness of the non-smooth solution component. A scalar parameter allows to balance between these two terms and, hence, to appropriately separate and regularize the smooth and non-smooth components of the solution. This paper proposes an efficient algorithm to solve this regularization problem by the alternating direction method of multipliers (ADMM). Furthermore, a novel algorithm for automatic choice of the balancing parameter is introduced, using robust statistics. The proposed approach is supported by some theoretical analysis, and numerical experiments concerned with different inverse problems are presented to validate the choice of the balancing parameter.

Maximum Entropy Technique and Regularization Functional for Determining the Pharmacokinetic Parameters in DCE-MRI

This paper aims to solve the arterial input function (AIF) determination in dynamic contrast-enhanced MRI (DCE-MRI), an important linear ill-posed inverse problem, using the maximum entropy technique (MET) and regularization functionals. In addition, estimating the pharmacokinetic parameters from a DCE-MR image investigations is an urgent need to obtain the precise information about the AIF–the concentration of the contrast agent on the left ventricular blood pool measured over time. For this reason, the main idea is to show how to find a unique solution of linear system of equations generally in the form of y=Ax+b, named an ill-conditioned linear system of equations after discretization of the integral equations, which appear in different tomographic image restoration and reconstruction issues. Here, a new algorithm is described to estimate an appropriate probability distribution function for AIF according to the MET and regularization functionals for the contrast agent concentration when applying Bayesian estimation approach to estimate two different pharmacokinetic parameters. Moreover, by using the proposed approach when analyzing simulated and real datasets of the breast tumors according to pharmacokinetic factors, it indicates that using Bayesian inference—that infer the uncertainties of the computed solutions, and specific knowledge of the noise and errors—combined with the regularization functional of the maximum entropy problem, improved the convergence behavior and led to more consistent morphological and functional statistics and results. Finally, in comparison to the proposed exponential distribution based on MET and Newton’s method, or Weibull distribution via the MET and teaching–learning-based optimization (MET/TLBO) in the previous studies, the family of Gamma and Erlang distributions estimated by the new algorithm are more appropriate and robust AIFs.

An unsupervised reconstruction method for low-dose CT using deep generative regularization prior

Low-dose CT imaging requires reconstruction from noisy indirect measurements which can be defined as an ill-posed linear inverse problem. In addition to conventional FBP method in CT imaging, recent compressed sensing based methods exploit handcrafted priors which are mostly simplistic and hard to determine. More recently, deep learning (DL) based methods have become popular in medical imaging field. In CT imaging, DL based methods try to learn a function that maps low-dose images to normal-dose images. Although the results of these methods are promising, their success mostly depends on the availability of high-quality massive datasets. In this study, we proposed a method that does not require any training data or a learning process. Our method exploits such an approach that deep convolutional neural networks (CNNs) generate patterns easier than the noise, therefore randomly initialized generative neural networks can be suitable priors to be used in regularizing the reconstruction. In the experiments, the proposed method is implemented with different loss function variants. Both analytical CT phantoms and human CT images are used with different views. Conventional FBP method, a popular iterative method (SART), and TV regularized SART are used in the comparisons. We demonstrated that our method with different loss function variants outperforms the other methods both qualitatively and quantitatively.

A hybrid regularization model for linear inverse problems

For the ill-posed linear inverse problem, we propose a hybrid regularization model, which possesses the characters of Tikhonov regularization and TV regularization to some extent. Through transformation, the hybrid regularization is reformulated as an equivalent minimization problem. To solve the minimization problem, we present two modified iterative shrinkage-thresholding algorithms (MISTA) based on the fast iterative shrinkage-thresholding algorithm (FISTA) and the iterative shrinkagethresholding algorithm (ISTA). The numerical experiments are performed to show the effectiveness and superiority of the regularization model and the presented algorithms.

Structured nonnegative matrix factorization for traffic flow estimation of large cloud networks

Network traffic matrix estimation is an ill-posed linear inverse problem: it requires to estimate the unobservable origin destination traffic flows, X, given the observable link traffic flows, Y, and a binary routing matrix, A, which are such that Y=AX. This is a challenging but vital problem as accurate estimation of OD flows is required for several network management tasks. In this paper, we propose a novel model for the network traffic matrix estimation problem which maps high-dimension OD flows to low-dimension latent flows with the following three constraints: (1) nonnegativity constraint on the estimated OD flows, (2) autoregression constraint that enables the proposed model to effectively capture temporal patterns of the OD flows, and (3) orthogonality constraint that ensures the mapping between low-dimensional latent flows and the corresponding link flows to be distance preserving. The parameters of the proposed model are estimated with a training algorithm based on Nesterov accelerated gradient and generally shows fast convergence. We validate the proposed traffic flow estimation model on two real backbone IP network datasets, namely Internet2 and GÉANT. Empirical results show that the proposed model outperforms the state-of-the-art models not only in terms of tracking the individual OD flows but also in terms of standard performance metrics. The proposed model is also found to be highly scalable compared to the existing state-of-the-art approaches.

On Hallucinations in Tomographic Image Reconstruction.

Tomographic image reconstruction is generally an ill-posed linear inverse problem. Such ill-posed inverse problems are typically regularized using prior knowledge of the sought-after object property. Recently, deep neural networks have been actively investigated for regularizing image reconstruction problems by learning a prior for the object properties from training images. However, an analysis of the prior information learned by these deep networks and their ability to generalize to data that may lie outside the training distribution is still being explored. An inaccurate prior might lead to false structures being hallucinated in the reconstructed image and that is a cause for serious concern in medical imaging. In this work, we propose to illustrate the effect of the prior imposed by a reconstruction method by decomposing the image estimate into generalized measurement and null components. The concept of a hallucination map is introduced for the general purpose of understanding the effect of the prior in regularized reconstruction methods. Numerical studies are conducted corresponding to a stylized tomographic imaging modality. The behavior of different reconstruction methods under the proposed formalism is discussed with the help of the numerical studies.

Quantifying Political Leaning from Tweets and Retweets

Media outlets and pundits have been quick to embrace online social networks to disseminate their own opinions. But pundits’ opinions and news coverage are often marked by a clear political bias, as widely evidenced during the fiercely contested2012 U.S. presidential elections. Given the wide availability of such data from sites like Twitter, a natural question is whether we can quantify the political leanings of media outlets using OSN data. In this work, by drawing a correspondence between tweeting and retweeting behavior, we formulate political leaning estimation as an ill-posed linear inverse problem. The result is a simple and scalable approach that does not require explicit knowledge of the network topology. We evaluate our method with a dataset of 119 million election-related tweets collected from April to November, and use it to study the political leaning of prominent tweeters and media sources.